Beyond Least Squares: Using Likelihoods

Setup

library(tidyverse)
library(tidymodels)
library(GGally)
library(knitr)
library(patchwork)
library(viridis)
library(ggfortify)

Learning goals

• Describe the concept of a likelihood

• Construct the likelihood for a simple model

• Define the Maximum Likelihood Estimate (MLE) and use it to answer an analysis question

• Identify three ways to calculate or approximate the MLE and apply these methods to find the MLE for a simple model

• Use likelihoods to compare models (next week)

Fouls in college basketball games

refs <- read_csv("data/04-refs.csv")
refs %>% slice(1:5) %>% kable()

We will treat the games as independent in this analysis.

Likelihood

Let \(p_H\) be the probability the referees call a foul on the home team.

The likelihood for a single observation

\[Lik(p_H) = p_H^{y_i}(1 - p_H)^{n_i - y_i}\]

Where \(y_i\) is the number of fouls called on the home team.

(In this example, we know \(n_i = 3\) for all observations.)

Model 1: Likelihood function

Because the observations (the games) are independent, the likelihood is

\[Lik(p_H) = \prod_{i=1}^{n}p_H^{y_i}(1 - p_H)^{3 - y_i}\]

We will use this function to find the maximum likelihood estimate (MLE). The MLE is the value between 0 and 1 where we are most likely to see the observed data.

Visualizing the likelihood

p <- seq(0,1, length.out = 100) #sequence of 100 values between 0 and 100
lik <- p^46 *(1 -p)^44

x <- tibble(p = p, lik = lik)
ggplot(data = x, aes(x = p, y = lik)) + 
  geom_point() + 
  geom_line() +
  labs(y = "Likelihood",
       title = "Likelihood of p_H")

Q: What is your best guess for the MLE, \(\hat{p}_H\)?

A. 0.489

B. 0.500

C. 0.511

D. 0.556

Finding the maximum likelihood estimate

There are three primary ways to find the MLE

Approximate MLE from a graph

Find the MLE using numerical approximation

Specify a finite set of possible values the for \(p_H\) and calculate the likelihood for each value

# write an R function for the likelihood
ref_lik <- function(ph) {
  ph^46 *(1 - ph)^44
}

# use the optimize function to find the MLE
optimize(ref_lik, interval = c(0,1), maximum = TRUE)

$maximum
[1] 0.5111132

$objective
[1] 8.25947e-28

Model 2: Conditional model

• Is there a tendency for the referees to call more fouls on the visiting team or home team?

• Is there a tendency for referees to call a foul on the team that already has more fouls?

Likelihood function

\[\begin{aligned}Lik(p_{H| N}, p_{H|H Bias}, p_{H |V Bias}) &= [(p_{H| N})^{25}(1 - p_{H|N})^{23}(p_{H| H Bias})^8 \\ &(1 - p_{H| H Bias})^{12}(p_{H| V Bias})^{13}(1-p_{H|V Bias})^9]\end{aligned}\]

(Note: The exponents sum to 90, the total number of fouls in the data)

Q: If fouls within a game are independent, how would you expect \(\hat{p}_H\), \(\hat{p}_{H\vert H Bias}\) and \(\hat{p}_{H\vert V Bias}\) to compare?

1. \(\hat{p}_H\) is greater than \(\hat{p}_{H\vert H Bias}\) and \(\hat{p}_{H \vert V Bias}\)

2. \(\hat{p}_{H\vert H Bias}\) is greater than \(\hat{p}_H\) and \(\hat{p}_{H \vert V Bias}\)

3. \(\hat{p}_{H\vert V Bias}\) is greater than \(\hat{p}_H\) and \(\hat{p}_{H \vert V Bias}\)

4. They are all approximately equal.

Q: If there is a tendency for referees to call a foul on the team that already has more fouls, how would you expect \(\hat{p}_H\) and \(\hat{p}_{H\vert H Bias}\) to compare?

1. \(\hat{p}_H\) is greater than \(\hat{p}_{H\vert H Bias}\)

2. \(\hat{p}_{H\vert H Bias}\) is greater than \(\hat{p}_H\)

3. They are approximately equal.

Acknowledgements

These slides are based on content in BMLR: Chapter 1 - Review of Multiple Linear Regression

Initial versions of the slides are by Dr. Maria Tackett, Duke University